Development of software applications typically involves an ongoing process of modifying the applications to add new functionality or features. As the applications are modified, an important goal of testing the modified applications is ensuring that existing features of the applications continue to perform as expected after modification. The process of testing applications to determine if the performance of existing features have changed, or regressed, is typically referred to as regression testing.
Regression testing for an application typically involves running test cases on the application before modification to determine its expected performance. Following modification, the same test cases can then be run on the application to determine if the application's performance has changed. Such regression testing requires recreating the same operating conditions for the application when the tests are run before and after modification. As applications become increasingly complex, the difficulty of recreating operating conditions for regression testing increases. For example, an application can interact with external software components (e.g., components not controlled by the application being tested) by sending data to or receiving data from the external software components. Recreating an operating environment in which these external software components behave the same and interact with the software application in the same way, with the same timing, across test case runs can create challenges for regression testing.
Conventional approaches to regression testing, whether automated or manual, often involve capturing the sequence of data sent by the external software components to the application when running test cases before modification. In these approaches, the same data is then provided to the application when running test cases after modification. However, these approaches do not account for all of the timing aspects necessary to ensure that the operating environment is the same for the application during pre-modification testing and post-modification testing.